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ABSTRACT i Se vf 
Attitude and opinion research requires considerable . 
planning, careful design of survey procedures and sampling . 
technigues, and practical ways of collecting, analyzing, and 
reporting results. The following suggestions are based on ‘the 
extensive public opinion research conducted by the University of 
Alabama. (1) Decide what specific probleg ‘is to be solved and whether 
a survey is the best way to solve it. (2) State the survey objectives 
clearly in writing. (3) Define the survey population. (4) AliSw 
enough time to|plan, conduct, and report the survey. (5) Identify and 
eliminate biases. (6) Be aware of, politics involved. (7) Do not 
hesitate to ask others for assistance. (8) Budget costs to assure an 
- accurate survey..(9) Recognize survey limitations. (10) Provide for 4 
"no opinion" responses. (11) Det raine advantages and disadvantages 5 * 
among mailed questionnaires, personal interviews, and telephone 
surveys. (12) Develop the survey jinstrument in terms of. the type of 
survey and the budget. (13) Detexmine the.survey procedure in terms 
of the entire population or a random sample. (14) Design a brief, 
specific, concise questionnaire using input from those to be 
surveyed. (15) Pre-test the questionnaire with groups similar to’. 
those’ to be surveyed. (16) Provide a\ stamped, self-addressed envelope 
with mailed questionnaires. (17) eee the results as soon as 
possible and release them. A selecte bibliography is ‘appended. 
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“Assessing Public Opinions “nd Attitudes 


_ Through Citizen Surveys* 
by 
, Thomas G. Acicea 


Institute of Higher Education Resea#th and Services 
_ The University of Alabama 


Much of ‘the opinion and attitude research that you,-and I 


regularly see offers some’ interesting facts ane figures * 
aeeut public attitudes'.and “opinions. And mandy times when we 
don't know what these attitudes and opinions are someone will 
invariably suggest "Let's conduct a poll" or "Let's take a 
survey." Whether the survey be in the form of a personal 
interview, a telephone call, a mail-out questionnaire or one 
of several Sintie years of collecting veuponsed: accurately 
neasievog nublic obinions and ‘attitudes is a delicate process 
“requiring donsiderabie planning, ‘careful design of survey 
procedures and sampling techniques, and erackidal ways of 
collecting, analeeina: and reporting results. In many cases, 
f however, results from surveys are iuenata or-unreliable because ~ 
the- novice researcher ‘does not know or fails to recognize many 
of the pitfalls of survey procedure. Hence the MERULEE may be 
biased, but worse than that, may actually nisrepresent pubite 


opirions and attitudgs. 
K 
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*Presentation to the ‘American Educational Research Association, _ 
annual meeting, ‘San Francisco, . California, April 8-12, 1979. : 


: . **Dr, Owings is an Associate in the Institute of Higher 
Education Research and Services and Adjunct Assistant Professor 


of Administration and oe Education at he University of 
Alabama. ; 
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In the ‘early 1970's the Institute of Higher. edudiraion 


‘Research and Services at The RAVER RT EY, of Alabama ‘initiated 


pesetenery planning th undertaking a Btatewsde survey designed re 
to accurately measure and assess. public ant nilene and attitudes : 


toward postsecondary educatten in Alabama. During the next 


Bole years, varying amounts of time were devoted to planning 


“for the ‘research PECTECe. In Fall, 1975, the research was 


undertaken as a full-time project of the THREES ane during - 


the next seven “Months development, of the questionnaire and 


design of the appropriate sampling ‘procedures were completed.. : 
The survey was conducted by mail nae 1976. From a 
modified random sample of 1251 A\abamians age 16 and over, 680 
usable eaplies were received resulting in a 54.5 percent response 


rate and yielding 4, Probable’ error of less than four percent 


‘ at the 95 percent confidence level. 


In order to provide comparative ‘data,. the survey was 


repeated in 1977, using the same procedures but drawing from a 


different sample... A few of the questions were changed or - 
modified but the najorjty of the questions were included in both | 
surveys. Again che sievey was conducted by mail in June-July, 
1977. From a modified random sample of 1259 Alabamians age 

16 and over, 655 usable questionnaires were received resulting 

in a 52.8 percent response rate and yielding a probable error 

"oF less’ than four percent at, the 95 percent confidence level. 
Although there were some minor changes in pbblic attitudes and 
opinions, no significant variations were noted. . 


. . If you are interested in the results of the two surveys and 


a summary of the 1977 survey I have copies of these reports. 
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If you would like either one or both see me after ah ng 


a Pe ; ze i “ , 
“presentation. : 


‘During the more than six years of planning dha conducting’ 
; <ene two citizens surveys, our. Institute staft/ gained - 
‘considerable ‘knowledge on design and procedures for ganducting 
opinion and attitude surveys as well as ways ‘tor reporting. 
‘and analyzing results. For the next few minutes let me share 
with zon suggestions for undertaking surveys and some of the 
pitfalls: to watch out for. . a | 

OFINION AND ATTITUDE RESEARCH: THERE IS A DIFFERENCE 

First of all, as Walter Lindenmann points out, there is 

a difference in Opsnton, and attitude research. Usually an 
attitude refers to the full range of impressions a person holds 
toward a social object including what the person gays, what the 
person knows and thinks, how the weaae Hexis; aud how the person 
‘is inclined to act. ‘In other words an attitude includes verbal, 
genet: emotional, and motivational ‘éxperiences. An opinion, 
however, usually refers only to what a person ‘says. The 
person's actions may or may not agree’ with his opinions. Since 
so many* factors make an attitude, measuring attitudes is cpmeae 
sophisticated. and complex and often expensive, while measuring ” ) 
an opinion is relatively’ simple and inexpensive. _ 

As Lindenmann points’ ‘out the methodology, involved in 
attitude. research relies on.a broad assortment of different 


- 


techniques including, the ‘survey, ‘che ri aabaaaie experiment, 


e 


various methods of. observa€ton, projactive tests, content analysis; 


‘ 


etc., while. opinion research usually relies on only one of these 


aes) 


a techniques--usually the survey. For purposes of discussion 


here, let's concentrate on opinion surveys since they are the 
: e 


_ principal method of research because of simplicity and costs. 


be ' (.. 
Here is a summary of suggestions I would make ‘to ‘persons 


-- interested in widivtaking a survey. 


', Opinion and attitude research is not for the novice 
researcher. College and university officials who have 
never done reserach in these areas would be better off 
consulting with experts or contracting with research 
agencies unless they can devote considerable time in 
Planning for the survey and in conducting the actual 

4 -research. 


. Decide what specific problem you are trying to solve and 
determine if a survey is the best way to find the : 


solution. If, for example, you want to know what 


certain people thigk about a given topic, you may choose : 
to survey them. If a survey is neither appropriate nor 
feasible, don't plan one. “. 


- State Our objectives clearly, \in writing, when -you 

» are thinking about conducting a survey. What is it you 
want to find out? What questions are absolutely necessary 
in determining what you want to find? From your list 
of objectives you can begin to move to other planning . 


phases of the survey. Objectives should be revised as 
/\ necessary. ‘ 


'. Who do you: want to survey? Define your universe., If, - 
or example you want to do an alumni survey for your ge le 
institution, then the universe (or defined populati'on) of 
your survey would be all the alumni of your institution. 
You can do a universe survey or a sample. If you do a ’ 
sample survey, be extremely sensitive as how you draw - 
the sample from your universe. A’ random/sample means : 
everyone in the universe (or your defined population) has 
an equal chance of being selected, A random sample does 
not mean stopping 20 people.on the street. to ask a few 
questions or calling 200° people from the telephone book. 
These are what I refer to as "haphazard" sampling. 


. Time. Allow enough time to plan, conduct,‘and report - 
your survey. Invariably this is my biggest weakness--I 
always underestimate the amount of time. You must allow id 
for unexpected aelare: 
° Biases. Bias ‘ts a forr ° letter eene and in research, 
especially opinion research, -it will jeopardize reliability 
and validity of the entire research project. In many 
cases the researcher will build biases into a study and 


o 


". - not even\ realize it. Try to look at all angles of the 
in order to aiaeaia! and eliminate biases. 


. Politics. 


research in this soe hon tebe remember that "once 
you open a\ can of worms, the only way to recan 
; them is to\use a larger can." 


- Help. Don’ \t be afraid to ask others for assistance, 
especiatly colleagues at your institution or other 
places who have special competencies in such areas as 
market research, management, tests and measurement, 
statistics, communications, languagé arts, political 
science, etc, 7 
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- Costs. Surveys cost money. Accurate surveys cost more % 
money. If you have neither the time nor the money 
‘to undertake.a project/that will give accurate and 


usable results| then for your own sake, don't undertake 
‘ it. If you're\going to use interviewers and pay them, 
be sure you include their pay in a budget and a plan 
for training them. If you expect.volunteer help then 
. be sure you know their competencies and skills .and 
don't overestimate their abilities. It's a common 
- problem to undertake a survey and half way through realize 
' that you're not going to have enough money to complete | 
it.. But because you've started, you feel your integrity 
is at stake and it must be completed come hell or high 
water. So you begin &Sutting back here and there and 
pretty soon, you"ve drawn a haphazard sample, are asking 
‘biased questions by using untrained, unskilled interviewers 
and are atte expecting to release a valid and reliable 
study. 


. Limitations. . Don't: forget that regardless of whatever 
research project you undertake you will have certain 
limitations. In opinion research learn to recognize 
and state the limitations of your survey. Obviously 
you)want to eliminate as many limitations as possible 
but/ the quality researcher will try to inélude in a 
study those factors which were either set or’over which = « 
he or she had no control, 


- No opinion responses. For almost every question in our 
' surveys we provided a "no opinion" or “not sure" choice 
_or both, unless wé wanted to force a positive or 
“negative response from those surveyed. By including 


these choices, you probably get a more accurate\ \idea of 
the state of public opinions. 


- Type of survey. There are several different methods of 
surveying people. Three commonly used methods are the 
mail-out questionnaire, the interview, and the telephone 
survey. All have peveneages clita alts ici a 


~ . e 


a 


wife 


A mail-out questionnaire is usually the cheapest 
kind. ‘It also has the lowest response rate; but by ~ 
using some techniques specialists have monitored over 
the years you can expect a response rate of 50 percent 
or more. Fifty’percent is:considered a good return 
rate. Specialists suggest that ‘hand-stamping--preferably 
with ‘a nice commemorative stamp--rather than. metering 
an envelope incredses response. In fact they suggest 
that the more expensive the postage the higher the ? 
response rate--all the way-up through air-mail special 
,delivery. Since you will have to calculate these factors, 
I Suggest using an advance notification card as Reader's 
Digest uses, informing the participant of his selection 
in the sample and soliciting his cooperation. I'll mention ~ 
more about the mail survey later on. 

Another method is the personal interview. An 
interview survey is somewhat more accurate and much more 
expensive. Interviewers must be trairied. If they 
aren't, people may tend to lie to them--not’ intentionally 
in most cases, but rather they tend to answer questions 
they way they think the interviewer wants them to be 
answered, rather than the way they real feel. An 
interview survey also permits: immediate’ follow-up to 
questions. Even though some people. will refuse to 
participate you should have a higher response rate than 
by mail questionnaires. Disadvantages are that ‘people 

. don't feel as' anonymous and the interviewers may make 
mistakes in recording answers.. 

A third commonly used survey method is ‘the teleghone 
interview. It fits somewhere between mail questionnaires 
and personal interviews as far as accuracy of.-response 

‘and costs. The survey method you select may be determined 
to a large extent = ‘the economic realities of the 
project. su 


_ « Survey Instrument. The survey instrument to some oxte® 
may be determined by the type of survey and the budget. 

Is it better to have a,closed (forced-choice) questionnaire 

or an open-ended questionnaire? Again there are 

advantages’ of each°and both are acceptable. ,It's 

easier to handle data from closed questionnaires. In 

any. event keep questions simple and to the point. 


- Survey procedure. As I mentioned before you can survey 
your entire universe or draw a sample. If you use random ° 
sampling, a very small sample can accurately reflect the 
opinions of your entire universe. As Cletis Pride points 
out in a paper entitled "Let's Take a Survey," the size 
of the sample does not have to increase proportionately 
with the size of the universe. If your universe is 200 
people, you should sample about 105. If your universe is 
5,000, a sample that giyes 210 responses will be adequate. 
If the universe doubles to about 10,000 your sample < * 
should be around 215. For 100, gue or so it should be 
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‘ : about 220. For most purposes you would rarely need 
more than 250 responses, provided your sample ies been 
_ scientifically sélected. 


**, 


- Random sampling. Selecting a random sample is not as 
difficult as you might think. Again as Pride points 
“out, let's asstime our. universe is 10,000 and we want a 
sample off 215. If we're doing a mail survey, we - 
should prabably select about. 450 names from our list 
in order to get about 215 responses. That means we're 
going to send a questionnaire to every 22nd person) in 
‘the universe.: In order to insure a random start, and. 
that's important, go to a table of random numbers (found 
in the back of any statistics book) and find the first 
number between one and 22. That will be the first name 
in your sample. Then add 22 to that number, take that 
nafe and so on ‘welt you've got all 450 names. 


. “Ways to_increase responses. If you use a mail-out | 
_ survey as we did, here are some suggestions to * 
increase responses: . 
a 


"* 1) Send an advance notification ‘card. . op Mie 


; 2) Enclose a stamped, self-addressed return envelope, 
— not a business reply envelope. The stamp, by the 
ane . % way, should be a regular, run-of-the mill postage - 
: : stamp and not a special commemorative stamp onthe 

‘return envelope. 


‘ 3) Design a iuieelinenks that’ s easy to complete sie 
preferably short. Involve those whom you are planning 
to survey in the pienning: PEeeees 


4) Always conduct a pilot survey (or PRenEeatlhs and 
solicit suggestions. ; 


5) Accompany the questionnaire with a letter of explanation 
signed by a prestigious individual. The letter should 
: explain to individuals why it is important for them 
ss to participate and that as participants tHey will 
A remain anonymous. ; 


Never put a code number on a questionnaire or ask for 

the respondents name, unless there are reasons for 

knowing who the respondent was. We devised a method 
for coding return envelopes which' we could check ‘off 

a master list as returned. 


7) Ask those demographic questions (you know the ones 
about age,, race, sex, etc.) at the end of the questionnaire 
rather than at the beginning. We found people would 
almost always answer those when at the end of the 
questionnaire. The one question that people most often 
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left blank or refused to answer was income, but even 
then, only less than 6% of the total number of 
=eApancents: failed to answer the question. 


mae f 8) Follow-up your first mailing. We sent out a 
t G - second letter, ‘another questionnaire, and stamped 
return envelope to those who hadn't responded, 
i... exactly two weeks after the first mailout. In 
- both of our surveys we had greater than 50 percent 
oe ¢ response rates. - ; \ 


9) Analyze your responses and get results out -as 
quickly as possible. The results of any survey, — 
especially an opinion survey should be released while 
they're still current. 


All you need to dd now is, add wont and worry. I have 
available one of the best. annotated bibliographies on current 
survey research that I have seen. It was prepared by 
Dr. Edward Gruson of the Sloan Commission on Government and . 
Higher Education at Cambridge, Massachusetts. Et you would 
like a copy please let me know. : 

. In summary , then, here's your survey checklist: . 7 
1) Define your problem. “Decide if a survey is the 
best road to a solution. ’ \ 
2) State the objectives of the curved in writing. 
3) Define your universe (or Boniabiont:. 
4) Decide’ on survey dioeeduey and size of’ the sample. 


5) Draw your sample using the random technique. Always 
protect the confidentiality of your survey respondents. 


6). Design the questionnaire 


a) ‘be brief 

b) be specific 

c) be clear and concise 
d) solicit input from those to be surveyed. 


7), Pre-test the questionnaire with Groups similat to 
those to be surveyed. 


’ 


. \ , ) 
8) Administer survey instrument. If it is a mail. 
questionnaire, be sure and include a stamped, self 
* 


es pea x = | 10 


oe ee Q a 
1 addressed return envelope and a letter of | F 
explanation, and mail a follow-up questionnaire 
two or three weeks after the first. If it is an- 


interview.or telephone survey be sure your interviewers 
are properly trained. 


9) Tabulate and analyze returns (use a computer ror: 
analysis and -cross-tabs). 


10) ‘Release results as soon as possible while Lear re 
still usable. 

And remember Gordon's First Law: 

a, : : 

"If a*research project is not worth doing 

at all, it is not worth doing well."' . » 


Sa 
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Erdos, Paul.L. Professional Mail Surve S. New York: xo 
McGraw-Hill, 1970. ° 


3 
An easy-to-follow text, by a recognized Saale in 
v market. ‘research. 
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* Haskins, J. B. "Pretesting Editorial Items and Ideas for 


Reader Interest.". _ Journalism Quarterly, 37 (Spring, 
1960), 224- Zans st 


Discusses alternatives to kinds of surveys. He used 
a sample of the target audience and rated their 
interest in titles. The assumption was a a reader. 
wdecides to read an article on the basis of its 
¥ apparent content. Members of the sample rate the 
titles (and/or subtitles) on a scale of 0-100 depending 
5 | on interest. If.a title gets a good rating, you use 
it: : 
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Lindenmann, Walter K. Attitude and Opinion Research: Why 


You Need 'It/How to Do It. Washington, D.C.: Council for 


Advancement and Support of Education, 1977, 83 pp.., 


A handbook designed to introduce the reader to opinion 
-. and attitude research, indicates the advantages of 

getting professional help, and tells how to do your 

own mail surveys. Also has nine sample questionnaires.. 


Parten, Mildred. Surveys, Polls and Samples: Practical 
Procedures. New York: Cooper Square Eta eaenone 1966. 


A how-to-do-it book first published 25 years ag0, but 
still a a regarded source. . 


Payne, Stanley. The Art of Askin Questions. Princeton, N.J.: 
Princeton University Breees 1951. . 
An excellent reference, ‘in spite of its age. Designed , 
to help the researcher ask questions in practical ways. 


Pride, Cletus. "Let' s Take a Survey." Techniques. 
Washington, D.C.: Council for the Advancement and Support 
 . Of Education, June, 1973, 4 pp. | . 


Summarizes how to conduct surveys, discusses alternate 
ways of conducting these inquiries, and makes suggestions 
on ways to increase responses ane analyze data, all in a 
brief and concise form. ‘ 
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